-
Notifications
You must be signed in to change notification settings - Fork 25.1k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add selector syntax to index expressions #118614
Merged
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
The logic is shimmed into place and will be updated in later commits to propagate its results to callers.
… to account for new method contracts.
Add logic for handling the selectors in alias filter collection.
…s are without selectors where possible.
@elasticmachine run buildkite/docs-build-pr |
This was referenced Jan 6, 2025
gmarouli
reviewed
Jan 7, 2025
...security/src/main/java/org/elasticsearch/xpack/security/authz/IndicesAndAliasesResolver.java
Show resolved
Hide resolved
gmarouli
reviewed
Jan 7, 2025
...security/src/main/java/org/elasticsearch/xpack/security/authz/IndicesAndAliasesResolver.java
Show resolved
Hide resolved
…ecurity/authz/IndicesAndAliasesResolver.java Co-authored-by: Mary Gouseti <mgouseti@gmail.com>
…ecurity/authz/IndicesAndAliasesResolver.java Co-authored-by: Mary Gouseti <mgouseti@gmail.com>
@elasticmachine run elasticsearch-ci/part-1 |
💔 Backport failed
You can use sqren/backport to manually backport by running |
💚 All backports created successfully
Questions ?Please refer to the Backport tool documentation |
jbaiera
added a commit
to jbaiera/elasticsearch
that referenced
this pull request
Jan 9, 2025
This PR introduces a new syntactical feature to index expression resolution: The selector. Selectors, denoted with a :: followed by a recognized suffix will allow users to specify which component of an index abstraction they would like to operate on within an API call. In this case, an index abstraction is a concrete index, data stream, or alias; Any abstraction that can be resolved to a set of indices/shards. We define a component of an index abstraction to be some searchable unit of the index abstraction. (cherry picked from commit c3839e1) # Conflicts: # modules/data-streams/src/internalClusterTest/java/org/elasticsearch/datastreams/IngestFailureStoreMetricsIT.java # server/src/main/java/org/elasticsearch/TransportVersions.java # server/src/test/java/org/elasticsearch/action/OriginalIndicesTests.java
jbaiera
added a commit
that referenced
this pull request
Jan 10, 2025
* Add selector syntax to index expressions (#118614) This PR introduces a new syntactical feature to index expression resolution: The selector. Selectors, denoted with a :: followed by a recognized suffix will allow users to specify which component of an index abstraction they would like to operate on within an API call. In this case, an index abstraction is a concrete index, data stream, or alias; Any abstraction that can be resolved to a set of indices/shards. We define a component of an index abstraction to be some searchable unit of the index abstraction. (cherry picked from commit c3839e1) # Conflicts: # modules/data-streams/src/internalClusterTest/java/org/elasticsearch/datastreams/IngestFailureStoreMetricsIT.java # server/src/main/java/org/elasticsearch/TransportVersions.java # server/src/test/java/org/elasticsearch/action/OriginalIndicesTests.java * [CI] Auto commit changes from spotless * Fixing compiler issues * Remove feature flag from influencing the serialisation * Only add failure indices when failure store flag is on * Fix OriginalIndicesTests * [CI] Auto commit changes from spotless --------- Co-authored-by: elasticsearchmachine <infra-root+elasticsearchmachine@elastic.co> Co-authored-by: Felix Barnsteiner <felixbarny@users.noreply.github.com> Co-authored-by: Mary Gouseti <mary.gouseti@elastic.co> Co-authored-by: Mary Gouseti <mgouseti@gmail.com>
This was referenced Jan 22, 2025
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Labels
auto-backport
Automatically create backport pull requests when merged
backport pending
:Data Management/Data streams
Data streams and their lifecycles
>non-issue
serverless-linked
Added by automation, don't add manually
Team:Data Management
Meta label for data/management team
v8.18.0
v9.0.0
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
In order to make adoption of failure stores simpler for all users, we are introducing a new syntactical feature to index expression resolution: The selector.
Selectors, denoted with a
::
followed by a recognized suffix will allow users to specify which component of an index abstraction they would like to operate on within an API call. In this case, an index abstraction is a concrete index, data stream, or alias; Any abstraction that can be resolved to a set of indices/shards. We define a component of an index abstraction to be some searchable unit of the index abstraction.To start, we will support two components:
data
andfailures
. Concrete indices are their owndata
components, while thedata
component for index aliases are all of the indices contained therein. For data streams, thedata
component corresponds to their backing indices. Data stream aliases mirror this, treating all backing indices of the data streams they correspond to as theirdata
component.The
failure
component is only supported by data streams and data stream aliases. Thefailure
component of these abstractions refer to the data streams' failure stores. Indices and index aliases do not have afailure
component.In order to select which components a user wants to operate on within an API call, the components are selected using the new selector syntax in an index expression:
On supporting APIs, selector syntax is evaluated first in every index expression. The syntax only accepts a predefined list of selector names or the match all
*
wildcard. It does not support any other selector names and it does not support simple match syntax. Any unsupported usage of the selector syntax will raise an exception.Examples:
Selector syntax is compatible with other expression syntax like date math and wildcard matching of index names:
Selector syntax will not be supported on every API as it does not always make sense. For example, the create and delete index APIs operate on indices only and will not support selectors. In these cases, any selector usage will raise an exception:
API's that support selectors will default to using the
::data
selector when selectors are not provided on an index expression. When evaluating expressions that have an explicit::data
selectors on them, the resolution code will translate the expressions back into names without selectors for consistency's sake.This PR was reconstructed from #113144 which it supercedes.